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Abstract  -  The  Modular  Ocean  Data  Assimilation 
System  (MODAS)  produces  oceanographic  nowcasts 
based  on  a)  climatology,  b)  remotely-sensed  sea  surface 
temperature  and  height,  and  c)  in-situ  measurements. 
Recent  analyses  have  shown  that  the  locations  of  in-situ 
measurements  can  have  a  profound  influence  on  the 
accuracy  of  the  MODAS  synthetic  profiles.  Small-scale 
variability  combined  with  sparse  sampling  and 
inappropriate  covariance  scales  can  lead  to  a  spreading  of 
unrepresentative  anomalies.  The  MODAS  Adaptive 
Sampling  Decision  Aid  (MASDA)  is  being  developed  to 
guide  the  selection  of  Airborne  Expendable  Bathy 
Thermograph  (AXBT)  measurement  locations  to  improve 
the  accuracy  of  MODAS  analyses  while  minimizing  the 
number  of  required  measurements.  MASDA  uses  the 
computed  MODAS  temperature  uncertainty  to  predict 
the  optimum  sampling  locations.  The  iterative  in-flight 
MASDA  approach  is  to  recommend  sequential 
measurement  locations  based  on  sequentially  computed 
temperature  uncertainty.  The  pre-flight  combinatorial 
MASDA  approach  is  to  recommend  the  best  combination 
of  N  measurement  locations  based  on  the  computed 
temperature  uncertainty.  These  environmentally  driven 
sampling  strategies  are  expected  to  increase  accuracy  of 
MODAS  analyses  relative  to  MODAS  analyses  based  on 
alternate  sampling  strategies  with  the  same  number  of 
observations.  AXBT  measurements  from  several  ocean 
areas  are  being  used  to  develop  and  test  MASDA 
algorithms.  Preliminary  results  showing  improvement  in 
MODAS  accuracy  using  the  MASDA  method  for  selecting 
observations  compared  to  more  subjective  selection 
methods  are  presented. 

1.  INTRODUCTION 

The  Modular  Ocean  Data  Assimilation  System 
(MODAS)  produces  oceanographic  nowcasts  based  on  a) 
climatology,  b)  remotely-sensed  sea  surface  temperature  and 
height,  and  c)  in-situ  measurements  [1-5].  Recent  analyses 
have  shown  that  the  locations  of  in-situ  measurements  can 
have  a  profound  influence  on  the  accuracy  of  the  MODAS 
synthetic  profiles.  Small-scale  variability  combined  with 
sparse  sampling  and  inappropriate  covariance  scales  can  lead 
to  a  spreading  of  unrepresentative  anomalies. 

The  MODAS  Adaptive  Sampling  Decision  Aid 
(MASDA)  is  being  developed  to  guide  the  selection  of 
Airborne  Expendable  Bathy  Thermograph  (AXBT) 
measurement  locations  to  improve  the  accuracy  of  MODAS 
analyses  while  minimizing  the  number  of  required 
measurements,  i.e.,  to  reduce  cost  and  time.  MASDA  uses 
the  computed  MODAS  temperature  uncertainty  to  predict  the 
optimum  sampling  locations.  The  iterative  in-flight  MASDA 
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approach  is  to  recommend  sequential  measurement  locations 
based  on  sequentially  computed  temperature  uncertainty,  i.e., 
choose  the  best  location,  assess  the  impact,  choose  the  next, 
etc.  The  pre-flight  combinatorial  MASDA  approach  is  to 
determine  the  best  combination  of  N  measurement  locations 
based  on  the  computed  temperature  uncertainty,  i.e.,  choose 
many  sets  of  N  locations,  then  assess  the  combined  impact  of 
all  N  locations  for  each  of  them,  and  then  select  the  best  set. 
By  focusing  on  reducing  uncertainty,  these  environmentally 
driven  sampling  strategies  are  expected  to  maximize  accuracy 
of  MODAS  analyses  relative  to  MODAS  analyses  based  on 
alternate  sampling  strategies  with  the  same  number  of 
observations. 

AXBT  measurements  from  several  ocean  areas  are  being 
used  to  develop  and  test  MASDA  algorithms.  In  the 
development  phase,  MASDA  is  constrained  to  select 
preferred  observation  points  from  among  the  set  of  AXBT 
measurement  locations.  After  the  MODAS  assimilation  of 
the  MASDA-selected  measurements,  the  MODAS 
temperatures  over  the  entire  set  of  measurement  locations  are 
compared  to  the  AXBT  measurements  to  determine  how 
accurate  the  resulting  MODAS  temperatures  are.  Preliminary 
results  show  improvement  in  MODAS  accuracy  using  the 
MASDA  method  for  selecting  observations  compared  to 
more  subjective  selection  methods. 

II.  MASDA 

MODAS  produces  oceanographic  nowcasts  and 
corresponding  oceanographic  uncertainty  nowcasts.  Initially 
the  MODAS  uncertainty  field  is  based  on  the  climatological 
standard  deviations  and  the  uncertainties  in  the  sea  surface 
temperature  and  height  projected  to  the  derived  synthetic 
profiles.  When  in-situ  measurements  are  ingested  by 
MODAS,  the  uncertainty  at  and  around  the  measurement 
locations  is  reduced  in  the  MODAS  nowcasts.  It  is  this 
ability  to  provide  temperature  uncertainty  that  makes 
MASDA  possible.  MASDA  uses  the  computed  MODAS 
temperature  uncertainty  to  predict  the  optimum  sampling 
locations  that  will  minimize  the  uncertainty  over  the  AXBT 
operational  region.  Since  we  can  a  priori  estimate  the 
uncertainty  (but  not  the  actual  error),  MASDA  assumes  that 
observations  that  minimize  uncertainty  also  tend  to  maximize 
accuracy  of  the  MODAS  analyses.  MASDA  incorporates 
two  approaches  to  predicting  the  optimum  sampling 
locations.  They  are  a)  the  iterative  in-flight  approach  and  b) 
the  pre-flight  combinatorial  approach. 
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A.  Iterative  In-flight  Approach 

The  iterative  in-flight  approach  starts  with  the  most  recent 
MODAS  uncertainty  field.  The  sequence  of  in-flight  steps 
follows.  1)  Evaluate  the  MODAS  uncertainty  field  to 
determine  the  location  of  highest  temperature  uncertainty. 
2)  Make  an  AXBT  measurement  at  that  location.  3) 
Assimilate  the  AXBT  measurement  into  MODAS  to  produce 
a  new  MODAS  field  and  associated  uncertainty.  4)  Evaluate 
this  new  uncertainty  field  to  determine  the  location  of  highest 
temperature  uncertainty.  5)  Repeat  steps  2  through  4.  This 
iterative  process  continues  until  the  available  AXBT  devices 
are  expended  or  the  uncertainty  reaches  an  acceptable  level. 
In  this  way,  MASDA  uses  information  from  previous 
measurements  to  avoid  over-sampling  the  environment  and  to 
improve  the  accuracy  of  MODAS  analyses  by  sampling  the 
environment  more  objectively  and  efficiently. 

As  MASDA  development  progressed,  it  became  apparent 
that  using  the  magnitude  of  MODAS  uncertainty  as  the  sole 
deciding  criterion  was  insufficient  or  inadequate.  Sometimes, 
the  MASDA-preferred  location  was  in  very  shallow  water. 
Although  the  uncertainty  was  highest  there,  little  temperature 
vs.  depth  information  would  be  obtained  by  a  measurement  at 
that  location,  limiting  the  relevance  of  that  observation  to 
estimates  over  the  upper  300-m.  Often  a  nearby  location  with 
slightly  less  uncertainty  but  in  deeper  water  would  provide 
more  temperature  vs.  depth  information  and  result  in  a  more 
accurate  MODAS  temperature  nowcast.  In  addition,  the 
defined  region  of  interest  may  cause  some  observations  to  be 
more  or  less  useful.  A  measurement  has  impact  on  the 
analysis  within  a  radius  of  its  location  as  defined  by  the 
MODAS  covariance  model,  which  for  this  study  is  based  on 
the  Rossby  radius  of  deformation.  Sometimes,  the  MASDA 
preferred  location,  based  on  maximum  uncertainty,  was  near 
the  coastline  or  the  edge  of  the  operational  area.  Such  a 
location  might  have  most  of  its  influence  outside  the  area  of 
interest.  Nearby  locations  away  from  the  boundary  provide 
information  relevant  to  larger  (useful)  regions.  Such 
locations  with  only  slightly  lower  uncertainty  can  be  expected 
to  more  efficiently  increase  overall  MODAS  nowcast 
accuracy. 

In  order  to  address  these  two  issues  of  water  depth  and 
area  of  impact,  water  volume  is  now  used  to  weight  the 
uncertainty  prior  to  selection  of  the  preferred  measurement 
location.  The  volume  is  a  function  of  the  water  depth  and  the 
Rossby  radius.  The  maximum  water  depth  is  set  to  300  m 
since  this  is  the  useful  depth  limit  for  AXBT  measurements. 
The  volume  does  not  include  land  or  extend  outside  the 
AXBT  operational  area.  By  weighting  the  uncertainty  by 
volume,  MASDA  can  select  locations  with  lower  absolute 
uncertainty  but  higher  volumetric  impact. 

B.  Pre-flight  Combinatorial  Approach 

The  second  MASDA  approach  is  the  pre-flight 
combinatorial  approach.  While  the  in-flight  method  targets 
locations  of  maximum  non-weighted  or  volume-weighted 
uncertainty,  the  pre-flight  method  calculates  the  impact  of  the 
observation(s)  on  the  uncertainty  of  the  MODAS  product. 


Like  the  iterative  in-flight  approach,  the  most  recent  MODAS 
uncertainty  field  is  used.  Unlike  the  iterative  in-flight 
approach,  the  pre-flight  combinatorial  version  of  MASDA  is 
run  before  the  AXBT  flight  leaves  the  ground.  The  user  must 
specify  the  number  of  AXBT  devices  to  be  dropped  and 
potential  measurement  locations.  MASDA  determines  every 
possible  combination  of  the  potential  measurement  locations 
with  the  specified  number  of  drops  and  the  total  volume- 
weighted  uncertainty  for  each  combination.  Overlapping 
volumes  for  locations  within  a  combination  are  divided 
equally  among  those  locations.  MASDA  then  selects  the  50 
combinations  with  highest  total  volume-weighted  uncertainty. 
For  each  of  those  combinations,  MODAS  is  run,  ingesting 
simulated  measurements  placed  at  the  measurement  locations 
within  the  combination,  to  determine  which  combination 
results  in  the  minimum  average  uncertainty  over  the  AXBT 
operational  region.  That  best  combination  of  AXBT 
measurement  locations  is  then  recommended. 

Although  the  pre-flight  combinatorial  approach  is  not 
designed  to  be  an  iterative  approach,  it  can  be  by  specifying 
one  AXBT  drop  and  running  MASDA  sequentially.  The 
result  using  the  pre-flight  combinatorial  approach  in  this  way 
is  not  necessarily  the  same  as  the  iterative  in-flight  approach. 
The  two  can  be  different  because  the  procedure  behind  each 
is  different.  For  the  next  measurement  location,  the  in-flight 
approach  picks  the  location  of  maximum  volume-weighted 
uncertainty  from  the  most  recent  MODAS  uncertainty  field. 
For  the  next  measurement  location,  the  iterative  pre-flight 
approach  picks  the  location  where  a  simulated  measurement 
ingested  into  MODAS  results  in  the  minimum  average 
uncertainty  over  the  AXBT  operational  region.  Those 
measurement  locations  may  not  be  same. 

III.  MASDA  RESULTS 

In  order  to  demonstrate  MASDA,  44  AXBT 
measurements  taken  in  the  summer  from  one  ocean  area  are 
used.  The  measurements  were  distributed  in  a  nearly  uniform 
pattern  throughout  the  area  of  interest  (outlined  in  white). 
Fig.  1  shows  the  bathymetry  for  this  region  with  shallow 
water  depths  in  the  West  and  deeper  water  depths  in  the  East. 
The  depth  is  indicated  by  color  with  the  dark  blue  color 
representing  depths  of  1800  m  and  greater. 

The  MODAS  temperature  uncertainty  based  on 
climatology  and  sea  surface  temperature  (SST)  is  shown  in 
Fig.  2.  The  darkest  blue  color  indicates  uncertainties  less 
than  1.50°C  and  the  darkest  red  color  indicates  uncertainties 
of  1.74°C  and  greater.  In  its  development  phase,  MASDA  is 
constrained  to  select  preferred  measurement  locations  from 
the  set  of  AXBT  measurement  locations.  The  star  indicates 
the  AXBT  location  that  is  chosen  as  the  first  measurement 
location  when  maximum,  unweighted  uncertainty  is  the 
deciding  criterion  for  MASDA.  Notice  that  this  location  is  a 
near-shore,  shallow  water  location.  It  does  have  the  highest 
uncertainty  but  a  measurement  at  this  location  will  provide 
limited  temperature-depth  information  and  have  limited 
regional  impact. 

The  water  volume  is  shown  is  Fig.  3.  The  volume  is  a 
function  of  the  Rossby  radius  (as  used  in  MODAS  covariance 
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Fig.  1.  Bathymetry  (m)  in  the  study  area. 
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Fig.  2.  MOD  AS  uncertainty  (°C)  in  the  study  area.  The  star 
indicates  the  AXBT  location  with  the  maximum  uncertainty. 
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Fig.  3.  Water  volume  (km  )  in  the  study  area. 
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Fig.  4.  Volume-weighted  uncertainty  (km3  *  °C)  in  the  study 
area.  The  star  indicates  the  AXBT  location  with  the 
maximum  volume -weighted  uncertainty. 
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model)  and  the  water  depth.  The  maximum  water  depth  used 
in  computing  the  volume  is  300  m.  The  darkest  blue  color 
indicates  volumes  of  less  than  1000  km3  and  the  darkest  red 
color  indicates  volumes  of  3400  km3  and  greater.  The 
volume  is  greatest  in  the  deeper  water  away  from  the 
operational  boundaries.  It  is  least  in  very  shallow  water  and 
near  the  corners  of  the  operational  area. 

The  volume-weighted  uncertainty  is  plotted  in  Fig.  4.  The 
darkest  blue  color  indicates  volume -weighted  uncertainty  is 
less  than  2100  (km3  *  °C)  and  the  darkest  red  color  indicates 
volume -weighted  uncertainty  is  greater  than  5500  (km3  *  °C). 
The  star  indicates  the  AXBT  location  chosen  for  the  first 
measurement  when  volume -weighting  is  applied.  The 
uncertainty  at  this  location  is  lower  than  that  of  the  location 
selected  by  the  unweighted  method  (1.69°C  vs.  1.74°C),  but 
the  greater  volume  more  than  compensates. 

Current  methods  for  selecting  AXBT  locations  are  highly 
subjective,  but  they  provide  the  only  benchmarks  available. 
To  assess  MASDA,  20  or  more  combinations  of  N  measure¬ 
ment  locations  were  hand-selected.  The  two  main  criteria  for 
the  hand  selection  were  to  1)  use  each  measurement  location 
in  at  least  one  combination  and,  more  importantly,  2)  spread 
the  observations  to  sample  the  entire  environment.  These 
selections  were  somewhat  biased  because  all  the  data, 
including  the  AXBT  measurements  and  previous  MODAS 
and  MASDA  results,  were  known  by  the  selector.  For  one 
observation,  each  AXBT  measurement  location  was  selected, 
so  there  are  44  combinations  with  one  observation.  There  are 
30  combinations  with  two  observations,  30  with  three,  20 
with  four,  and  30  with  five. 

The  average  uncertainty  as  a  function  of  the  number  of 
observations  is  shown  in  Fig.  5.  The  uncertainty  is  averaged 
over  the  first  100-m  of  depth  and  then  all  44  AXBT 
measurement  locations.  The  average  uncertainty  for  zero 
observations  is  the  average  of  the  climatology  and  SST 
uncertainties  at  the  AXBT  locations.  The  purple  dots 
(labeled  “Select”)  indicate  the  uncertainties  for  the  hand- 
selected  combinations.  When  uncertainty  alone  is  the 
deciding  criterion  by  which  the  iterative  in-flight  MASDA 
selects  preferred  locations,  the  blue  line  (labeled  “Uncert”) 
shows  the  result.  With  each  added  observation,  the  average 
uncertainty  decreases  but  hand-selecting  the  measurement 
locations  is  normally  better. 

When  volume -weighted  uncertainty  is  the  deciding 
criterion  by  which  the  iterative  in-flight  MASDA  selects 
preferred  locations,  the  green  line  (labeled  “Volume”)  in  Fig. 
5  results.  Again,  the  average  uncertainty  decreases  as  the 
number  of  observations  increase.  For  the  first  five 
observations,  the  average  uncertainty  is  less  with  volume¬ 
weighting  than  without.  For  one  and  two  observations,  the 
volume-weighted  MASDA  approach  results  in  uncertainties 
equal  to  or  less  than  the  minimum  uncertainty  of  the  hand- 
selected  combinations  (“Select”).  For  three  or  more 
observations,  there  are  hand-selected  combinations  with 
lower  average  uncertainty.  For  the  iterative  approach,  each 
single  selection  is  optimal  for  reducing  uncertainty  from  the 
step  immediately  prior,  but  an  alternate  deployment  (one 
which  produces  sub-optimal  intermediate  reductions  in 


uncertainty)  may  ultimately  lead  to  greater  uncertainty 
reduction. 

The  red  stars  (labeled  “Combo”)  are  the  pre-flight 
combinatorial  MASDA  results.  Since  there  are  only  44 
AXBT  locations,  there  are  only  44  combinations  with  one 
observation.  For  multiple  observations,  there  are  50 
combinations  each.  Data  from  MASDA-selected  locations 
are  ingested  into  MODAS  to  determine  which  combination 
results  in  the  minimum  average  uncertainty,  as  shown  in  Fig. 
5.  In  cases  with  more  than  one  observation,  the  pre-flight 
combinatorial  MASDA  produced  sets  of  observations  which 
reduced  uncertainty  more  than  the  other  methods  (“Select”, 
“Uncert”,  and  “Volume”). 

The  pre-flight  combinatorial  MASDA  approach  can  be 
run  in  an  iterative  fashion  by  specifying  one  location  at  a  time 
and  running  MASDA  sequentially.  The  result  is  shown  in 
Fig.  5  as  a  black  line  (labeled  “Iterate”).  When  MASDA  is 
used  in  this  fashion,  the  minimum  uncertainty  is  not  reduced 
for  three  and  more  observations  as  much  as  the  combinatorial 
planning  method.  However,  the  minimum  uncertainty  is  the 
same  or  less  than  that  for  both  the  unweighted  uncertainty 
(“Uncert”)  and  volume-weighted  uncertainty  (“Volume”) 
iterative  in-flight  MASDA.  Table  I  lists  the  average 
uncertainty  for  all  these  cases.  The  hand-selected  uncertainty 
listed  in  the  table  is  the  minimum  of  the  average  hand- 
selected  uncertainties.  The  minimum,  rather  than  the 
average,  of  the  average  hand-selected  uncertainties  was 
chosen  for  consistency. 


TABLE  I 

Average  uncertainty  vs.  number  of  observations 


Number  of 
Observations 

Select 

In-flight  MASDA 

Pre-flight  MASDA 

Uncert  |  Volume 

Combo  |  Iterate 

0 

!!  1.675  | 

1 

1.467 

1.522 

1.467 

1.467 

1.467 

2 

1.326 

1.425 

1.321 

1.318 

1.318 

3 

1.238 

1.331 

1.256 

1.203 

1.232 

4 

1.153 

1.286 

1.218 

1.132 

1.160 

5 

1.087 

1.197 

1.165 

1.072 

1.094 

Building  confidence  in  MODAS  by  optimally  reducing 
uncertainty  is  one  goal  of  this  work.  Another  goal  is  to 
reduce  absolute  temperature  differences  between  the 
MODAS  and  AXBT  temperatures.  The  average  RMS  error 
as  a  function  of  the  number  of  observations  is  shown  in  Fig. 
6.  The  RMS  error  is  averaged  over  the  first  300-m  of  depth 
and  then  all  44  AXBT  locations.  The  AXBT  measurements 
are  compared  to  the  MODAS  first  guess  temperatures  (clima¬ 
tology  with  SST  assimilation)  to  determine  the  average  RMS 
error  for  zero  observations.  The  format  of  this  figure  is 
similar  to  the  previous  figure.  The  errors  for  the  hand- 
selected  combinations  with  the  minimum  average  uncertainty 
are  outlined  in  black.  The  minimum  average  uncertainty  was 
chosen  for  consistency.  For  the  first  two  observations,  they 
are  over-plotted  by  the  pre-flight  combinatorial  MASDA 
errors. 

Table  II  summarizes  these  results.  For  the  in-flight 
MASDA  (“Uncert”  and  “Volume”),  there  is  no  consistent 
improvement  in  error  reduction  when  the  volume-weighting 
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Number  of  Observations 

Fig.  5.  Average  uncertainty  (°C)  vs.  number  of  observations. 


Number  of  Observations 

Fig.  6.  Average  RMS  error  (°C)  vs.  number  of  observations. 


is  applied  prior  to  selection  of  preferred  locations.  The  pre- 
flight  iterative  errors  (“Iterate”)  are  similar  to  the  in-flight 
errors  (“Uncert”  and  “Volume”).  The  non-iterative,  pre¬ 
flight  combinatorial  MASDA  approach  (“Combo”)  is  the  best 
overall  MASDA  technique  to  date  for  reducing  error, 
especially  for  3  or  more  observations. 


TABLE  II 

Average  RMS  error  vs.  number  of  observations 


Number  of 
Observations 

Select 

In-flight  MASDA 

Pre-flight  MASDA 

Uncert  |  Volume 

Combo  |  Iterate 

0 

2.129 

1 

1.949 

2.106 

1.949 

1.949 

1.949 

2 

1.711 

1.663 

1.813 

1.713 

1.713 

3 

1.459 

1.570 

1.528 

1.311 

1.517 

4 

1.479 

1.560 

1.445 

1.299 

1.497 

5 

1.271 

1.547 

1.369 

1.305 

1.470 

Minimum  average  uncertainty  does  not  necessarily  equate 
to  minimum  average  RMS  error.  If  the  true  error  field  were 
known  beforehand,  implying  that  the  true  temperature  field  is 
known,  there  would  be  no  question  of  where  the  measure¬ 
ments  (if  any)  should  be  made  and  MASDA  would  not  be 
needed.  The  MODAS  uncertainty  is  the  best  guide  available 
at  this  time  to  help  select  measurement  locations  to  improve 
MODAS  nowcasts.  As  MASDA  development  progress, 
MASDA  will  continue  to  be  tested  in  the  various  ocean  areas. 
The  MASDA  pre-flight  combinatorial  approach  still  needs 
further  software  development  and  evaluation.  As  the  number 


of  measurements  to  be  made  increases,  the  task  of  computing 
all  the  combinations  becomes  unreasonable.  If  further  testing 
confirms  that  the  uncertainty  reduction  using  the  pre-flight 
combinatorial  MASDA  approach  in  an  iterative  fashion  is 
comparable  to  the  uncertainty  reduction  using  the  pre-flight 
combinatorial  MASDA  approach  in  its  original  non-iterative 
fashion,  it  may  be  the  best  solution. 

IV.  SUMMARY 

Data  from  several  ocean  areas  were  used  to  develop  an 
approach  for  optimal  AXBT  sampling  in  support  of  MODAS 
nowcasts.  The  MASDA  algorithms  for  selection  of  optimum 
sampling  locations  were  examined  in  the  study  area  and 
evaluated  in  terms  of  reduced  temperature  uncertainty  and 
increased  MODAS  accuracy.  The  in-flight  MASDA 
approach  recommends  sequential  measurement  locations 
based  on  sequentially  computed  MODAS  temperature 
uncertainty.  The  pre-flight,  planning  MASDA  approach 
recommends  the  best  combination  of  N  measurement 
locations,  also  based  on  MODAS  temperature  uncertainty. 
Volume  weights  on  the  uncertainty  estimates  improve 
MASDA  AXBT  location  selections  based  on  reduced  average 
uncertainty.  Associated  error  reduction  is  greatest  when  the 
pre-flight  combinatorial  method  is  used.  These  preliminary 
results  are  encouraging  and  are  leading  to  better  algorithms 
for  choosing  sampling  locations  to  maximize  MODAS 
accuracy  and  minimize  the  number  of  required  samples. 
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